Consistent Variance Estimator Research Articles

Enhanced Laplace approximation

The Laplace approximation has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators derived from the Laplace approximation are often biased for binary or temporally and/or spatially correlated data. Additionally, the corresponding Hessian matrix tends to underestimates the standard errors of these approximate maximum likelihood estimators. While higher-order approximations have been suggested, they are not applicable to complex models, such as correlated random effects models, and fail to provide consistent variance estimators. In this paper, we propose an enhanced Laplace approximation that provides the true maximum likelihood estimator and its consistent variance estimator. We study its relationship with the variational Bayes method. We also define a new restricted maximum likelihood estimator for estimating dispersion parameters and study their asymptotic properties. Enhanced Laplace approximation generally demonstrates how to obtain the true restricted maximum likelihood estimators and their variance estimators. Our numerical studies indicate that the enhanced Laplace approximation provides a satisfactory maximum likelihood estimator and restricted maximum likelihood estimator, as well as their variance estimators in the frequentist perspective. The maximum likelihood estimator and restricted maximum likelihood estimator can be also interpreted as the posterior mode and marginal posterior mode under flat priors, respectively. Furthermore, we present some comparisons with Bayesian procedures under different priors.

Augmented inverse probability weighted estimation and prediction for cause-specific proportional hazards regression with missing covariates

This paper describes estimation of the regression parameters and prediction of the cumulative incidence functions under the cause-specific proportional hazards model when some of covariates are not fully observed. Assuming that missingness mechanism is missing at random, we propose the augmented inverse probability weighted method for estimation and inference procedures. A nonparametric regression approach is adapted for estimating selection probabilities and conditional expectations of missing covariates in the augmented estimating function. We establish the asymptotic properties of the predicted cumulative incidence functions under the cause-specific proportional hazards model with missing covariates and derive consistent variance estimators of the predicted cumulative incidence functions. Simulation studies show that the procedures perform well. The proposed methods are illustrated with stage IV breast cancer data obtained from the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute.

M-estimation for common epidemiological measures: introduction and applied examples.

M-estimation is a statistical procedure that is particularly advantageous for some comon epidemiological analyses, including approaches to estimate an adjusted marginal risk contrast (i.e. inverse probability weighting and g-computation) and data fusion. In such settings, maximum likelihood variance estimates are not consistent. Thus, epidemiologists often resort to bootstrap to estimate the variance. In contrast, M-estimation allows for consistent variance estimates in these settings without requiring the computational complexity of the bootstrap. In this paper, we introduce M-estimation and provide four illustrative examples of implementation along with software code in multiple languages. M-estimation is a flexible and computationally efficient estimation procedure that is a powerful addition to the epidemiologist's toolbox.

Identification and estimation of causal peer effects using double negative controls for unmeasured network confounding.

Identification and estimation of causal peer effects are challenging in observational studies for two reasons. The first is the identification challenge due to unmeasured network confounding, for example, homophily bias and contextual confounding. The second is network dependence of observations. We establish a framework that leverages a pair of negative control outcome and exposure variables (double negative controls) to non-parametrically identify causal peer effects in the presence of unmeasured network confounding. We then propose a generalised method of moments estimator and establish its consistency and asymptotic normality under an assumption about ψ-network dependence. Finally, we provide a consistent variance estimator.

Inference in Linear Dyadic Data Models with Network Spillovers

Abstract When using dyadic data (i.e., data indexed by pairs of units), researchers typically assume a linear model, estimate it using Ordinary Least Squares, and conduct inference using “dyadic-robust” variance estimators. The latter assumes that dyads are uncorrelated if they do not share a common unit (e.g., if the same individual is not present in both pairs of data). We show that this assumption does not hold in many empirical applications because indirect links may exist due to network connections, generating correlated outcomes. Hence, “dyadic-robust” estimators can be biased in such situations. We develop a consistent variance estimator for such contexts by leveraging results in network statistics. Our estimator has good finite-sample properties in simulations, while allowing for decay in spillover effects. We illustrate our message with an application to politicians’ voting behavior when they are seating neighbors in the European Parliament.

Out-of-sample error estimation for M-estimators with convex penalty

Abstract A generic out-of-sample error estimate is proposed for $M$-estimators regularized with a convex penalty in high-dimensional linear regression where $(\boldsymbol{X},\boldsymbol{y})$ is observed and the dimension $p$ and sample size $n$ are of the same order. The out-of-sample error estimate enjoys a relative error of order $n^{-1/2}$ in a linear model with Gaussian covariates and independent noise, either non-asymptotically when $p/n\le \gamma $ or asymptotically in the high-dimensional asymptotic regime $p/n\to \gamma ^{\prime}\in (0,\infty )$. General differentiable loss functions $\rho $ are allowed provided that the derivative of the loss is 1-Lipschitz; this includes the least-squares loss as well as robust losses such as the Huber loss and its smoothed versions. The validity of the out-of-sample error estimate holds either under a strong convexity assumption, or for the L1-penalized Huber M-estimator and the Lasso under a sparsity assumption and a bound on the number of contaminated observations. For the square loss and in the absence of corruption in the response, the results additionally yield $n^{-1/2}$-consistent estimates of the noise variance and of the generalization error. This generalizes, to arbitrary convex penalty and arbitrary covariance, estimates that were previously known for the Lasso.

Frequentist analysis of basket trials with one-sample Mantel-Haenszel procedures.

Recent substantial advances of molecular targeted oncology drug development is requiring new paradigms for early-phase clinical trial methodologies to enable us to evaluate efficacy of several subtypes simultaneously and efficiently. The concept of the basket trial is getting of much attention to realize this requirement borrowing information across subtypes, which are called baskets. Bayesian approach is a natural approach to this end and indeed the majority of the existing proposals relies on it. On the other hand, it required complicated modeling and may not necessarily control the type 1 error probabilities at the nominal level. In this article, we develop a purely frequentist approach for basket trials based on one-sample Mantel-Haenszel procedure relying on a very simple idea for borrowing information under the common treatment effect assumption over baskets. We show that the proposed Mantel-Haenszel estimator for the treatment effect is consistent under two limiting models of the large strata and sparse data limiting models (dually consistent) and propose dually consistent variance estimators. The proposed estimators are interpretable even if the common treatment effect assumptions are violated. Then, we can design basket trials in a confirmatory matter. We also propose an information criterion approach to identify effective subclasses of baskets.

Assumption-light and computationally cheap inference on inequality measures by sample splitting: the Student t approach

Inference on inequality indices remains challenging, even in large samples. Heavy right tails in income and wealth distributions hinder the quality and threaten the validity of asymptotic approximations to finite sample distributions. Attempts to improve on asymptotic approximations by bootstrap techniques or permutation tests are only partial successes. We evaluate a different approach to robust inference, relying on Student t statistics obtained from split samples. This relatively simple ‘t-based’ approach requires no consistent variance estimators, no random sampling of populations, and only mild distributional assumptions. We compare its performance with that of refined bootstrap and permutation techniques. We find that the more complex bootstrap methods still have the edge in one-sample tests, where the t-approach suffers from a negative skew. In two-sample comparisons though, the t-approach offers advantages: it is undersized while bootstrap tests and permutation tests are often oversized. In certain circumstances it is less powerful than permutation tests and bootstrap tests, but for large samples, this difference dissipates. It is also more generally applicable than permutation tests and easily generates confidence intervals. These differences are illustrated with an empirical application using two different sources of household data from the Russian Federation.

Estimating SARS-CoV-2 seroprevalence.

Governments and public health authorities use seroprevalence studies to guide responses to the COVID-19 pandemic. Seroprevalence surveys estimate the proportion of individuals who have detectable SARS-CoV-2 antibodies. However, serologic assays are prone to misclassification error, and non-probability sampling may induce selection bias. In this paper, non-parametric and parametric seroprevalence estimators are considered that address both challenges by leveraging validation data and assuming equal probabilities of sample inclusion within covariate-defined strata. Both estimators are shown to be consistent and asymptotically normal, and consistent variance estimators are derived. Simulation studies are presented comparing the estimators over a range of scenarios. The methods are used to estimate severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) seroprevalence in New York City, Belgium, and North Carolina.

Spatial Linear Regression with Covariate Measurement Errors: Inference and Scalable Computation in a Functional Modeling Approach

For spatial linear models, the classical maximum-likelihood estimators of both regression coefficients and variance components can be biased when the covariates are measured with errors. This work introduces a theoretically backed-up estimation framework for the spatial linear errors-in-variables model in a functional approach. Compared with the structural models, the functional approach treats the unobserved true covariates as fixed unknown parameters without imposing additional structures, thus leading to more robust parameter inference. Our model parameters are estimated simultaneously based on a set of unbiased estimating equations. Under some regularity conditions, we prove the consistency of the proposed estimating-equation estimators and derive their asymptotic distribution. In addition, a consistent variance estimator is developed for the estimating-equation estimators. To handle large spatial datasets, we provide two approaches to obtain scalable estimations based on our proposed estimating equations, where the required computational time and storage are reduced to be linear with sample size for each estimating-function evaluation. Simulation studies under different settings show that our estimators are consistent and the scalable algorithms work well. Finally, the proposed method is applied to studying the relationship between Arctic sea ice and related geophysical variables. Supplementary materials for this article are available online.

Improved inference for doubly robust estimators of heterogeneous treatment effects.

We propose a doubly robust approach to characterizing treatment effect heterogeneity in observational studies. We develop a frequentist inferential procedure that utilizes posterior distributions for both the propensity score and outcome regression models to provide valid inference on the conditional average treatment effect even when high-dimensional or nonparametric models are used. We show that our approach leads to conservative inference in finite samples or under model misspecification and provides a consistent variance estimator when both models are correctly specified. In simulations, we illustrate the utility of these results in difficult settings such as high-dimensional covariate spaces or highly flexible models for the propensity score and outcome regression. Lastly, we analyze environmental exposure data from NHANES to identify how the effects of these exposures vary by subject-levelcharacteristics.

A subsampling perspective for extending the validity of state-of-the-art bootstraps in the frequency domain

Summary Bootstrapping spectral mean statistics has been a notoriously difficult problem over the past 25 years. Many frequency domain bootstraps are valid only for certain time series structures, e.g., linear processes, or for special types of statistics, i.e., ratio statistics, because such bootstraps fail to capture the limiting variance of spectral statistics in general settings. We address this issue with a different form of resampling, namely, subsampling. While not considered previously, subsampling provides consistent variance estimation under much weaker conditions than any existing bootstrap in the frequency domain. Mixing is not used, as is often standard with subsampling. Rather, subsampling can be generally justified under the same conditions needed for original spectral mean statistics to have distributional limits in the first place. This result has impacts for other bootstrap methods. Subsampling then applies to extending the validity of recent state-of-the-art bootstraps in the frequency domain. We nontrivially link subsampling to such bootstraps, which broadens their range, as moment and block assumptions needed for these are cut by more than half. Essentially, state-of-the-art bootstraps then require no more stringent assumptions than those needed for a target limit distribution to exist, which is unusual in the bootstrap world. We also close a gap in the theory of subsampling for time series with distributional approximations, in addition to variance estimation, for frequency domain statistics.

Signal selection for local module identification in linear dynamic networks: A graphical approach

In a dynamic network of interconnected transfer functions, it is not necessary to use all the node signals for estimating a local transfer function. Given the network topology, detailed conditions are available for selecting inputs and outputs in a (MIMO) predictor model that warrants consistent and minimum variance estimation of a target module through the so-called local direct method. Motivated by the existing minimum-input signal selection approach that gradually incorporates additional signals, an alternative graphical algorithm for signal selection is developed in this work by directly exploiting the complete network graph. Then, as a straightforward application of existing analytical results, graphical conditions for consistent identification are derived for the novel signal selection approach. We show by an example that in some cases, for the consistent estimation of the target module, the developed method leads to fewer selected signals than the original minimum-input method.

Genetic and Environmental Variation in Continuous Phenotypes in the ABCD Study®

Twin studies yield valuable insights into the sources of variation, covariation and causation in human traits. The ABCD Study® (abcdstudy.org) was designed to take advantage of four universities known for their twin research, neuroimaging, population-based sampling, and expertise in genetic epidemiology so that representative twin studies could be performed. In this paper we use the twin data to: (i) provide initial estimates of heritability for the wide range of phenotypes assessed in the ABCD Study using a consistent direct variance estimation approach, assuring that both data and methodology are sound; and (ii) provide an online resource for researchers that can serve as a reference point for future behavior genetic studies of this publicly available dataset. Data were analyzed from 772 pairs of twins aged 9–10 years at study inception, with zygosity determined using genotypic data, recruited and assessed at four twin hub sites. The online tool provides twin correlations and both standardized and unstandardized estimates of additive genetic, and environmental variation for 14,500 continuously distributed phenotypic features, including: structural and functional neuroimaging, neurocognition, personality, psychopathology, substance use propensity, physical, and environmental trait variables. The estimates were obtained using an unconstrained variance approach, so they can be incorporated directly into meta-analyses without upwardly biasing aggregate estimates. The results indicated broad consistency with prior literature where available and provided novel estimates for phenotypes without prior twin studies or those assessed at different ages. Effects of site, self-identified race/ethnicity, age and sex were statistically controlled. Results from genetic modeling of all 53,172 continuous variables, including 38,672 functional MRI variables, will be accessible via the user-friendly open-access web interface we have established, and will be updated as new data are released from the ABCD Study. This paper provides an overview of the initial results from the twin study embedded within the ABCD Study, an introduction to the primary research domains in the ABCD study and twin methodology, and an evaluation of the initial findings with a focus on data quality and suitability for future behavior genetic studies using the ABCD dataset. The broad introductory material is provided in recognition of the multidisciplinary appeal of the ABCD Study. While this paper focuses on univariate analyses, we emphasize the opportunities for multivariate, developmental and causal analyses, as well as those evaluating heterogeneity by key moderators such as sex, demographic factors and genetic background.

Sensitivity analyses in longitudinal clinical trials via distributional imputation.

Missing data is inevitable in longitudinal clinical trials. Conventionally, the missing at random assumption is assumed to handle missingness, which however is unverifiable empirically. Thus, sensitivity analyses are critically important to assess the robustness of the study conclusions against untestable assumptions. Toward this end, regulatory agencies and the pharmaceutical industry use sensitivity models such as return-to-baseline, control-based, and washout imputation, following the ICH E9(R1) guidance. Multiple imputation is popular in sensitivity analyses; however, it may be inefficient and result in an unsatisfying interval estimation by Rubin's combining rule. We propose distributional imputation in sensitivity analysis, which imputes each missing value by samples from its target imputation model given the observed data. Drawn on the idea of Monte Carlo integration, the distributional imputation estimator solves the mean estimating equations of the imputed dataset. It is fully efficient with theoretical guarantees. Moreover, we propose weighted bootstrap to obtain a consistent variance estimator, taking into account the variabilities due to model parameter estimation and target parameter estimation. The superiority of the distributional imputation framework is validated in the simulation study and an antidepressant longitudinal clinical trial.

Regression analysis for covariate-adaptive randomization: Arobust and efficient inference perspective.

Linear regression is arguably the most fundamental statistical model; however, the validity of its use in randomized clinical trials, despite being common practice, has never been crystal clear, particularly when stratified or covariate-adaptive randomization is used. In this article, we investigate several of the most intuitive and commonly used regression models for estimating and inferring the treatment effect in randomized clinical trials. By allowing the regression model to be arbitrarily misspecified, we demonstrate that all these regression-based estimators robustly estimate the treatment effect, albeit with possibly different efficiency. We also propose consistent non-parametric variance estimators and compare their performances to those of the model-based variance estimators that are readily available in standard statistical software. Based on the results and taking into account both theoretical efficiency and practical feasibility, we make recommendations for the effective use of regression under various scenarios. For equal allocation, it suffices to use the regression adjustment for the stratum covariates and additional baseline covariates, if available, with the usual ordinary-least-squares variance estimator. For unequal allocation, regression with treatment-by-covariate interactions should be used, together with our proposed variance estimators. These recommendations apply to simple and stratified randomization, and minimization, among others. We hope this work helps to clarify and promote the usage of regression in randomized clinical trials.

Lasso-adjusted treatment effect estimation under covariate-adaptive randomization

Summary We consider the problem of estimating and inferring treatment effects in randomized experiments. In practice, stratified randomization, or more generally, covariate-adaptive randomization, is routinely used in the design stage to balance treatment allocations with respect to a few variables that are most relevant to the outcomes. Then, regression is performed in the analysis stage to adjust the remaining imbalances to yield more efficient treatment effect estimators. Building upon and unifying recent results obtained for ordinary-least-squares adjusted estimators under covariate-adaptive randomization, this paper presents a general theory of regression adjustment that allows for model mis-specification and the presence of a large number of baseline covariates. We exemplify the theory on two lasso-adjusted treatment effect estimators, both of which are optimal in their respective classes. In addition, nonparametric consistent variance estimators are proposed to facilitate valid inferences, which work irrespective of the specific randomization methods used. The robustness and improved efficiency of the proposed estimators are demonstrated through numerical studies.

Competing risks predictions with different time scales under the additive risk model.

In the analysis for competing risks data, regression modeling of the cause-specific hazard functions has been usually conducted using the same time scale for all event types. However, when the true time scale is different for each event type, it would be appropriate to specify regression models for the cause-specific hazards on different time scales for different event types. Often, the proportional hazards model has been used for regression modeling of the cause-specific hazard functions. However, the proportionality assumption may not be appropriate in practice. In this article, we consider the additive risk model as an alternative to the proportional hazards model. We propose predictions of the cumulative incidence functions under the cause-specific additive risk models employing different time scales for different event types. We establish the consistency and asymptotic normality of the predicted cumulative incidence functions under the cause-specific additive risk models specified on different time scales using empirical processes and derive consistent variance estimators of the predicted cumulative incidence functions. Through simulation studies, we show that the proposed prediction methods perform well. We illustrate the methods using stage III breast cancer data obtained from the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute.

Inference in latent factor regression with clusterable features

Regression models, in which the observed features X∈Rp and the response Y∈R depend, jointly, on a lower dimensional, unobserved, latent vector Z∈RK, with K≪p, are popular in a large array of applications, and mainly used for predicting a response from correlated features. In contrast, methodology and theory for inference on the regression coefficient β∈RK relating Y to Z are scarce, since typically the un-observable factor Z is hard to interpret. Furthermore, the determination of the asymptotic variance of an estimator of β is a long-standing problem, with solutions known only in a few particular cases. To address some of these outstanding questions, we develop inferential tools for β in a class of factor regression models in which the observed features are signed mixtures of the latent factors. The model specifications are both practically desirable, in a large array of applications, render interpretability to the components of Z, and are sufficient for parameter identifiability. Without assuming that the number of latent factors K or the structure of the mixture is known in advance, we construct computationally efficient estimators of β, along with estimators of other important model parameters. We benchmark the rate of convergence of β by first establishing its ℓ2-norm minimax lower bound, and show that our proposed estimator βˆ is minimax-rate adaptive. Our main contribution is the provision of a unified analysis of the component-wise Gaussian asymptotic distribution of βˆ and, especially, the derivation of a closed form expression of its asymptotic variance, together with consistent variance estimators. The resulting inferential tools can be used when both K and p are independent of the sample size n, and also when both, or either, p and K vary with n, while allowing for p>n. This complements the only asymptotic normality results obtained for a particular case of the model under consideration, in the regime K=O(1) and p→∞, but without a variance estimate. As an application, we provide, within our model specifications, a statistical platform for inference in regression on latent cluster centers, thereby increasing the scope of our theoretical results. We benchmark the newly developed methodology on a recently collected data set for the study of the effectiveness of a new SIV vaccine. Our analysis enables the determination of the top latent antibody-centric mechanisms associated with the vaccine response.

Lpdensity: Local Polynomial Density Estimation and Inference

Density estimation and inference methods are widely used in empirical work. When the underlying distribution has compact support, conventional kernel-based density estimators are no longer consistent near or at the boundary because of their well-known boundary bias. Alternative smoothing methods are available to handle boundary points in density estimation, but they all require additional tuning parameter choices or other typically ad hoc modifications depending on the evaluation point and/or approach considered. This article discusses the R and Stata package lpdensity implementing a novel local polynomial density estimator proposed and studied in Cattaneo, Jansson, and Ma (2020, 2022), which is boundary adaptive and involves only one tuning parameter. The methods implemented also cover local polynomial estimation of the cumulative distribution function and density derivatives. In addition to point estimation and graphical procedures, the package offers consistent variance estimators, mean squared error optimal bandwidth selection, robust bias-corrected inference, and confidence bands construction, among other features. A comparison with other density estimation packages available in R using a Monte Carlo experiment is provided.